DOCUMENT RESUME 



ED 447 206 

AUTHOR 

TITLE 



PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 
ABSTRACT 

as part of the Eisenhower Initial Teacher Professional Development Program in 
Oregon, under the Proficiency-based Admissions Standards System (PASS) . STEP 
focused on the development of a framework of teacher knowledge, skills, and 
materials for pre-service and practicing teachers. High school teachers and 
university faculty across six disciplines committed one or more classes to 
the implementation of standards -based instruction and assessment. In 
collaborative work groups they shared pedagogical practices, broadened their 
content knowledge, and learned new ways to engage students in learning. This 
paper summarizes the findings of the project and the culminating state 
institutes in the spring of each year that provided the opportunity for 
participants to share information. Data were collected over a 2 -year period 
for 44 secondary school teachers and college faculty. Findings from the STEP 
study suggest that teachers engaged in the actual implementation of standards 
provide clear imperatives for those charged with the formulation of 
educational policy and the design of teacher preparation and professional 
development programs. The major findings of this study are informing the next 
steps in the development of standards-based reform in Oregon. Fifteen areas 
of teacher practice were identified and grouped into the categories of 
target/plan, teach/assess, and verify. The framework of teacher practice 
these categories define will be further refined for inclusion in teacher 
preparation and licensure programs in Oregon. (Contains 52 references.) (SLD) 



TM 032 114 

Tell, Christine A.; Bodone, Francoise M. ; Addie, Karen L. 

A Framework of Teacher Knowledge and Skills Necessary in a 
Standards -Based System: Lessons from High School and 
University Faculty. 

2000-04-28 

43p . ; Paper presented at the Annual Meeting of the American 
Educational Research Association (New Orleans, LA, April 
24-28, 2000) . 

Reports - Evaluative (142) -- Speeches/Meeting Papers (150) 

MF01/PC02 Plus Postage. 

♦Academic Standards; *College Faculty; Cooperation; High 
Schools; Higher Education; *Knowledge Base for Teaching; 
♦Models; Program Evaluation; *Secondary School Teachers; 
♦Teaching Methods 
Oregon 

The Standards-based Education Project (STEP) was conducted 




Reproductions supplied by EDRS are the best that can be made 
from the original document. 



>r 



VO 

O 

<N 

■'3' 

■'3' 



Q 



W 



A Framework of Teacher Knowledge and Skills Necessary in a 

Standards-Based System: 

Lessons from High School and University Faculty 



Christine A. Tell Frangoise M. Bodone 

christine_tell@ous.edu francoise_bodone@ous.edu 



Karen L. Addie 
karen_addie@ous.edu 



Proficiency-based Admission Standards System (PASS) 
Oregon University System 
P.O. Box 3175 
Eugene, OR 97403-0175 
(541) 346-5799 
http://www.ous.edu/pass 



Paper presented at the 

American Educational Research Association Annual Meeting 

New Orleans 





04 

CO 

© 





© 2000, April 28 



SL .. J CENTER (ERIC) 
^-Sieivedlrnm !h as been re P'°«uc e d as 

0^4 i. P6rS0n ° r ° r9a " izati °" 

□ Minor changes have been made to 
improve reproduction quality. 



Pomts of view or opinions stated in this 

official OEn? n0, .. necessari| y represent 
ornciai OERI position or policy. 



2 



BEST COPY AVAILABLE 



PERMISSION TO REPRODUCE AND 

disseminate this material has 

BEEN GRANTED BY 

XL- tell 



TO THE EDUCATIONAL RESOURCES 
^ INFORMATION CENTER (ERIC) 



1 



A Framework of Teacher Knowledge and Skills Necessary in a Standards-based System: 
Lessons from High School and University Faculty. 



INTRODUCTION AND CONTEXT 

States involved in educational reform are now confronting the challenge of how teachers 
might best be prepared to insure that students achieve the standards that have been adopted. This 
paper presents the findings from a two-year study aimed at identifying the core knowledge and 
skills necessary to teach in a standards-based system. The Standards-based Education Project 
(STEP) was conducted as part of the Eisenhower Initial Teacher Professional Development 
Program and focused on the development of a framework of teacher knowledge, skills, and 
materials for pre-service and practicing teachers. High school teachers and university faculty 
across six disciplines committed one or more classes to the implementation of standards-based 
instruction and assessment. In collaborative work groups, they shared pedagogical practices, 
broadened their content knowledge, and learned new ways to engage students in learning. 
Culminating statewide institutes in the spring of each year provided an opportunity for 
participants to share classroom-performance assessments and determine levels of student 
proficiency. This paper summarizes these findings and suggests a framework for the 
consideration of teacher educators, professional developers, and educational researchers who are 
involved in standards-based reform. 

State and National Policies Dictate the Need for a Framework 
of Teacher Knowledge and Skills 

Oregon is similar to many other states that have been engaged in nearly two decades of 
educational reform. Various attempts to introduce academic rigor statewide have lead to the 
development of common curriculum goals, minimum competencies for student performance, and 
“world class standards” designed to produce a quality workforce for the emerging global 
economy. The initiative that endured beyond the rhetoric was the Educational Act for the 21 st 
Century, which was passed in 1991 and amended in 1995. The Act mandated the implementation 
of a comprehensive system of content and performance standards and assessments benchmarked 
at grades 3, 5, 8 and 10. By demonstrating proficiency on state tests and classroom performance 
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assessments that are aligned with the standards, students earn Certificates of Initial and 
Advanced Mastery (CIM at grade 10, CAM at grade 12+). State-sponsored Goals 2000 grants 
initially encouraged school districts to pioneer unique solutions to the implementation of 
standards at the local level. Changes in the school funding structure and the creation of the 
statewide assessment system shifted resources and discretionary grant programs to the attainment 
of state standards. 

Similar to other states, Oregon has adopted a K-16 perspective with the goal of creating a 
seamless educational system. However, Oregon retains the unique distinction of being the only 
state in the nation with state board-adopted standards that align student performance in 
elementary, middle, and high school with college admissions. In 1994, the Oregon University 
System (OUS) adopted a policy to develop the Proficiency-based Admission Standards System 
(PASS). PASS links admission to the state’s seven public universities with the new K- 12 
standards and assessments. Our STEP study was connected to PASS and sought to answer the 
question, What knowledge and skills are necessary for teaching in a standards-based system? 

While the policy framework in Oregon provides the broad brushstrokes for a standards- 

\ 

based system, the aligned K- 1 6 standards and resulting assessment system create a complex 
mandate for classroom teachers as agents of implementation. At the classroom level, the culture 
of standards and performance assessments challenges traditional teacher practice and moves the 
once-private act of judging student performance into the public arena (Tell, Bodone, and Addie, 
1999). Teacher judgment of student proficiency via various local- and state-mandated 
performance assessments plays an equally critical role requiring higher levels of teacher 
competence in the design of instruction and assessment. v. 

Oregon’s need for a highly skilled and knowledgeable teaching force is also reflected as a 
national challenge in the Report of the National Commission on Teaching and America’s Future 
(1996). In this report. Governor Hunt of North Carolina noted that “standards for students and 
teachers are the key to reforming American education. Access to competent teaching must 
become a new student right. The reform movement of the last decade cannot succeed unless it 
attends to the improvement of teaching” (p. iii). Our study provides insight into what constitutes 
competent teaching in a standards-based system. 

Federal legislation subsequent to this National Commission Report further suggested that 
teacher education institutions be held accountable for preparing teachers for a standards-based 
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system. As one of the first round of states receiving federal funding under Title II, Oregon has 
undertaken the dual challenge of redesigning teacher education programs across 1 6 public and 
private institutions and of creating a reporting and accountability system to document the results. 

Oregon’s teacher-quality initiative builds on a statewide policy framework. In 1997, the 
Oregon State Legislature unanimously enacted Senate Bill 124 with the support of the Teachers 
Standards and Practices Commission (TSPC — the state licensing agency), the Joint Boards of 
Education, and educational leaders across the state. The new design includes three-year initial 
and five-year continuing teaching licenses. The practicing teacher has a significant role in 
designing and documenting a Continuing Professional Development Plan (CPD) that addresses 
six domains of professional competence. The CPD, along with successful teaching, provides 
evidence to initiate and renew teaching licenses. Finally, Oregon’s Senate Bill 880 further links 
student learning to teacher performance by replacing guarantees of tenure with two-year 
contracts and renewals based on proven ability in standards-based teaching (TSPC, 1999). 

Our study links to a key accountability requirement of Oregon’s teacher-education 
redesign - that teachers be able to demonstrate their ability to design instruction and assessment 
that address the standards and document their students’ performance relative to those standards. 
Underlying this requirement is the question, What teacher knowledge and skills are needed to 
insure student learning in a standards-based system? The findings of our two-year study of the 
practice of teachers and university faculty who were engaged in standards-based teaching begin 
to respond to this complex question. 

Throughout this series of reforms at the state and national level, the need for higher 
standards in the preparation of teachers has become obvious, since “what teachers know and can 
do makes the crucial difference in what children learn” (NCTAF Report, 1996, p.5). The idea of 
a knowledge base for teachers (Dill, 1990) and the importance of teacher professionalization has 
emerged over the last several years and generated a considerable amount of rhetoric and 
discussion (Cawelti, 1995; Darling-Hammond and Sykes, 1999; Dill, 1990; Goodlad, 1990; 
McLaughlin and Oberman, 1996; Wiske, 1998). The significance of our study is that its findings 
about teacher knowledge emerged from a standards-based classroom learning environment. 
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Practitioners’ Experiences Serve as a Referent for Policy Implementation in Oregon 



The Standards-based Teacher Education Project (STEP) was undertaken by the Oregon 
University System (OUS) to determine the knowledge and skills necessary to teach in a 
standards-based system. OUS is the coordinating body for Oregon’s seven public universities, all 
but one of which house teacher-preparation programs. Throughout the development of the 
Proficiency-based Admission Standards System (PASS), OUS relied on the experience of higher 
education faculty and on teachers from high schools that were serving as research and 
development sites. While the design of PASS required extensive teacher and faculty input, it 
soon became apparent that its actual implementation would require changes in classroom 
practice. Our two-year study involved 44 teachers and- faculty who were involved in this 
development effort. Our intent was that the actual experience of practitioners engaged in 
implementation would inform the further development of teacher-preparation programs and 
materials, continuing professional development, and policies related to standards 
implementation. 

Using practitioner experience as a referent for policy implementation is both pragmatic 
and practical. As the agents of implementation, teachers are held to high levels of accountability 
for successful student performance in ambitious systemic reform efforts such as Oregon’s. 
Policymakers may fail to acknowledge teachers as initiators of change who are able to define 
problems and devise solutions (Tyack and Cuban, 1995; Schulman and Sykes, 1983). The 
exclusion of such voices is both perplexing and problematic. When they view teachers simply as 
“conduit[sj for instructional policy, not as actor[sj” policymakers tend to “invest a great deal 
more in the creation of control systems for teaching than they do in the development of teacher 
knowledge” (Darling-Hammond, 1 990, p. 345). By contrast, enlisting the support and skills of 
teachers as key actors in reform might be seen as a positive kind of modeling, adapting 
knowledgeably to local needs and circumstances by preserving what is valuable and correcting 
what is not (Tyack and Cuban, 1995). 

A one-dimensional “top-down” approach to policy implementation invites failure by 
disengagement (McLaughlin, 1 987). As one STEP high school social science teacher aptly 
stated, “Never underestimate the ability of teachers to stonewall or sabotage change imposed. 

We can all adopt a new terminology, series of reporting forms, and then go back to our 
classrooms, close our doors, and do exactly what we’ve always done.” Even though teachers 
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may have full discretion over decision making within their classroom walls, they are often 
excluded from management decisions beyond these walls (Shedd and Bacharach, 1991). In 
essence, teachers may be relegated to the status of “lower participants” in the formal 
organization of their school, whose voices are “difficult to access” (LeCompte, 1993, p.10) by 
those who are in positions of authority. Or, as Gitlin and Meyers (1993) suggest: 

Teachers’ stories are rarely given a forum in the public domain. This is not 
particularly surprising given that teachers often are not expected to formulate 
policy or influence the direction of educational practice but rather are expected to 
take the wisdom of others and find ways to put these theories into practice, (pp. 

51-52) 

In stark contrast to this culture of disengagement, our study provided a forum for the 
voices of teachers engaged in the implementation of standards (Bodone, 1997). It was reasoned 
that teachers, through careful study of their own practice, could clearly articulate the knowledge 
and skills needed for teaching in a standards-based classroom. The “legitimate knowledge” 
(Gitlin et al., 1 992) generated through this reflective and collaborative work could then best 
inform the design of teacher-preparation programs, professional development, and policies 
guiding teacher education and accountability. 



METHODOLOGY 



Research Conducted “With,” not “On” Practitioners 



Our study uses qualitative methods while drawing on principles of both action research 
and educative research to elicit participants’ voices and document their experiences. A process 
similar to that suggested by McNiff (1988) for action research, “involving a self-reflective spiral 
of planning, acting, observing, reflecting, and re-planning” (p. 7), provided a conceptual frame 
for the initial stages of participants’ inquiry. Similar to Gitlin’s (1992) notion of educative 
research, this study “broadens the notion of expertise by valuing participants’ experiential 
knowledge as well as the knowledge produced through their systematic inquiry” (p. 2). The 
educative process is one in which participants become responsible for the very research that is 
being done on their practice and are subsequently accountable for the understanding and 
transformation of their practice (Bodone, 1997). Through the use of action research and 




Tell, Bodone, and Addie (2000). A Framework of Teacher Knowledge and Skills Necessary in a Standards-based System: 
Lessons from High School Teachers and University Faculty 



7 



- u ^ 



6 



educative research over a two-year period, participants were able to move beyond simple 
reflection upon their own practice to critical reflection and action — what Freire (1970) called 
“praxis.” 



Participants Organized into Co-development Teams 

The data were collected over a two-year period, from the fall of 1996 to the summer of 
1998. Forty-four secondary teachers and higher education faculty participated in this reflective 
study of professional practice. These voluntary participants were from a purposeful sample 
(Patton, 1990) representing 17 high schools and five higher education institutions. These schools 
were geographically and demographically representative of many areas in the state of Oregon. 
The project teachers were selected and invited to participate because they were already members 
of the larger PASS project and therefore experienced with standards-based reform. 

The teachers and faculty were organized into co-development teams of three to eight 
members representing six disciplines: English, mathematics, science, social science, second 
languages, and visual and performing arts. Each co-development team was comprised of 
beginning (2-5 years) and veteran (up to 30 years) teachers who worked in both rural and urban 
settings serving diverse student populations and communities. The teacher and faculty participant 
groups reflected diverse ethnic backgrounds, personal characteristics, and professional 
experiences. This enabled us to access a multitude of descriptive “teachers’ voices” rather than a 
more singular, prescriptive “teacher’s voice” (Hargreaves, 1996). 

The high school and higher education co-development teams were a unique feature of our 
study, bringing two sectors of education that rarely interface together into collaborative work 
groups. The teams provided opportunities for focused conversations on standards-based teaching 
and learning, the sharing of instruction and assessment materials, and the informal and formal 
evaluation of student work. Participants were treated as teams of experts, each of whom was 
engaged in the implementation of a standards-based K-16 system and contributed to the wisdom 
gleaned from practice as a teacher - whether beginning, veteran, high school, or higher 
education. 
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Inquiry Focused on Knowledge and Skills 

The participants’ reflective study began within the context of an action research cycle 
focused on the knowledge and skills necessary to teach in a standards-based system. Initially, the 
inquiry revolved around the question of what knowledge and skills are necessary for teaching in 
a standards-based system? The members of each co-development team were free to select any 
one of a number of PASS proficiency standards in their content areas (math, English, science, 
social science, visual and performing arts, and second languages) as a focus for instruction and 
assessment. Each participant captured their experiences in a journal and collected classroom 
artifacts (assignments, assessments, and student work) to illustrate standards-based practices. 
Meeting four to five times a year as co-development teams and once as an entire STEP project 
group, participants’ learning emerged in these general thematic areas: Teacher knowledge and 
skills, instructional strategies, student learning, and shifts in thinking. 

At the end of the first year, focus groups conducted with each co-development team 
examined specific practices, such as how teachers determine course content, select instructional 
strategies, and evaluate student progress. General reflections included what impact standards 
have on teaching behavior and students’ learning. 

During the second year, participants in each of the six content area co-development teams 
elected to adopt common standard(s) for student learning. The PASS proficiency standard(s) that 
were selected addressed: (a) writing and literary analysis (English); (b) problem-solving in a sub- 
discipline such as algebra (mathematics); (c) scientific inquiry in a sub-discipline such as biology 
(science); (d) performance within a social/historical context in the arts such as music (visual and 
performing arts); (e) social science analysis in a sub-discipline such as history (social science); 
and (f) oral proficiency and written expression (second languages). Selecting a common standard 
created a common context for sharing teacher materials and student work at team meetings, and 
for assembling collections of student work throughout the year for scoring at the statewide PASS 
Verification Institute. 

Participants also began the second year of the study with a common framework for their 
inquiry. This framework was organized around a general set of teacher practices in a standards- 
based system. The practices, which emerged from the first-year data, included: Targeting 
specific standards for teaching and learning; Planning instruction and assessment around that 
target; Teaching with a variety of strategies to ensure student progress toward that standard; 
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Assessing student learning across time with multiple measures; Verifying individual judgments 
of student proficiency with other team members through the cross-scoring of student work; and 
Reflecting upon individual experiences. When taken together, the practices suggest a cycle 
(Figure 1) throughout which the teacher focuses on the student learning suggested by the 
standard. 



Figure 1: Cycle of Teaching Practices in a Standards-based Classroom 




Using co-development team questions (Table 1) derived from the first year's data and 
corresponding to the cycle, participants were more focused. As a result, they delved further into 
their own pedagogical practices, specifically describing things that seemed to enhance specific 
student learning. 
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Table 1 : STEP Co-Development Team Questions for Year 2 



Target 


• What do you think (the standard) expects students to know and be able to do? 

• What types of information will you use to determine what students need to 
know and be able to do? 


Plan 


• How will you plan to assess students’ knowledge and skills? 

• How will you plan to teach (the standard)? 

• How will you help students make meaning of (the standard)? 

• What are the knowledge and skills students will need to study the standard? 

• How will you assess students’ prior knowledge and skills related to the 
standard? 

• What specific content knowledge and skills will you need to teach to and assess 
(the standard)? 


Teach 


• What instructional methods/strategies/resources will you use to teach (the 
standard) in your classroom? 

• How will you connect (the standard) to what students know and where they 
need to be to meet the standard? 

• How will you approach the varying levels of knowledge and skills among your 
students? 


Assess 


• How will you know if a student is “proficient” or not? 

• What evidence will you use to determine this? 


Verify 


• How does your judgment of student proficiency compare with other teachers’ 
judgments of that student’s work? 

• How will you know that you share a common understanding of student 
proficiency in (the standard)? 

• How will you work with colleagues to develop a common understanding of (the 
standard), review and cross-score student work, and verify judgments about 
student proficiency? 


Reflect 


• How do you use evidence of student performance to improve your professional 
practice? 

• How will you modify, refine or renew practices in designing curriculum, 
instruction and assessment? 

• How will you further your own content knowledge and pedagogy? 

• What role will colleagues play in furthering your professional development? 
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Sources of Data 

Multiple methods of data collection were used to capture each co-development team’s 
experiences as standards implementation revolved around an action-research cycle. The sources 
of data collected include the following items: 

1 . Journal entries - these included general reflections on standards-based teaching within the 
context of class(es) in which teachers agreed to focus instruction on a specific standard, assess 
student performance toward that standard, and generate student work that would then be shared 
with other teachers in the group along with classroom tasks, scoring guides, and other materials. 
Journals used hand written or electronic recording and were shared during four of the group 
meetings that occurred between September and June of the first year. 

2. Focus eroup responses - conducted with each of the six discipline groups in June of the first year. 
Questions focused on recommendations for teacher education programs based on participants' 
reflections of their teaching in a standards-based classroom during the year. 

3. Co-development teams’ responses to focused questions - derived from the journal process of the 
first year, these 20 questions in six categories (target, plan teach, assess, verify, reflect - Table I) 
directed our conversations to a cycle of classroom activities. 1 Specific questions were assigned 
via email and summarized for subsequent group meetings. These questions then served as a point 
of departure for further discussion. Group meetings occurred every other month from September 
through June of the second year. 

4. Co-development team proceedings - during year one, participants and facilitators had open- 
ended meeting discussions addressing all aspects of their standards-based implementation 
experience. Colleagues brought teacher and student materials (i.e. scoring guides, tasks, 
curriculum plans, student-parent communication, etc.) to share, peer critique, and co-develop. In 
the second year the proceedings were guided by a more structured agenda, including the 
revisiting of email question responses, focus questions for discussion, materials sharing, and 
teachers’ questions and issues as they work with standards. Individual co-development teams 
met four to five times a year at various locations. The meetings convened for two- to five-hour 
sessions and all proceedings were audio taped and eventually transcribed. 



1 This cycle led to a "Verification Institute" in which all participating teachers met in their content-area groups and 
verified or confirmed one another's judgments on proficiency (or lack thereof) based on their students’ work. 
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5. Teacher materials - Through the duration of the study, participants in each group shared 
classroom tasks, activities, scoring guides, and samples of student work. From the discussion of 
these materials, they derived strategies and techniques for improving their practice(s) in a 
standards-based classroom. 

6. Teacher surveys - the STEP high school teachers were a subset of the approximately 100 PASS 
teachers from 18 Oregon high schools who completed surveys in the spring of 1997 and 1998. 
The instrument examined teacher knowledge and levels of use of standards. The analysis of 
responses also included possible differences between STEP/non-STEP teachers as a group and 
by content area. Higher education faculty participating in STEP did not complete the surveys. 

7. Student surveys - these surveys were administered in the same PASS sites and at the same time 
as were the teacher surveys, and analyzed in a similar fashion. 

The project director and two research assistants systematically collected data using the 
various methods outlined above for all six of the co-development teams in each content area 
(math, English, science, social science, visual and performing arts, and second languages). The 
two assistants also coordinated and facilitated the co-development team meetings, with each 
assistant supporting three groups. All meetings, focus groups and face-to-face interviews were 
audio taped and transcribed. The hand-written journal entries were also transcribed, and the 
email responses from participants were prepared for coding. 

Studying multiple participants and institutional settings and employing multiple methods 
of data collection over time allowed for triangulation of the data, and in doing so greatly 
strengthened the study’s usefulness for other settings (Marshall and Rossman, 1995). The 
importance of multiple sources in order to provide and maintain a reliable chain of evidence 
about the topic in question was highlighted by Yin (1994). He suggests that these multiple 
sources essentially provide “multiple measures of the same phenomena” (p. 92). The major 
purpose of the open-ended focus group and individual interviews was to "learn to see the world 
from the eyes of the person being interviewed" (Ely, Anzul, Friedman, Gamer and Steinmetz, 
1991). Seidman (1991) wrote: "Interviewing provides access to the context of people's behavior 
and thereby provides a way for researchers to understand the meaning of that behavior . . . [It] 
allows us to put behavior in context and provide access to understanding their action" (p. 4). The 
focus group interviews and meetings encouraged expression of various points of view, therefore 
providing participants with an opportunity to hear opinions and understandings and then form 
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their own (Marshall and Rossman, 1995). The teacher materials shared during team meetings 
corroborated information collected through journals, meetings, focus-group and individual 
interviews, and email question responses (Yin, 1994). 

Data Analysis 

The project director and the two research assistants systematically and collaboratively 
analyzed the data that were collected from all the various sources across the six co-development 
teams. The raw data from year one were formatted and prepared for entry into a database that 
was organized for the analysis software Qualitative Solutions and Research Non-numerical 
Unstructured Data Indexing Searching and Theorizing (QSR Nud*ist). This software package 
aided us with our analysis by supporting the processes of coding data in an index system, 
searching text, and searching patterns for coding and theorizing about the data (Gahan and 
Hannibal, 1998). 

Next, a system of content analysis that involved identifying, coding, and categorizing the 
main data themes was adopted. This was a preliminary categorizing of participant responses. 

The program enabled us to code the data based on these categories and, after coding it, to 
examine the categories for patterns across the six content teams and within the individual groups. 
As we analyzed the data from year one, we looked for “reoccurring regularities in the data” 
(Patton, 1990) that produced patterns which could then be sorted into categories. Participants’ 
response patterns across the six content areas were coded into a total of 24 emergent categories 
framed with the following titles: student learning, teacher behaviors, instructional strategies, and 
shifts in thinking. We then worked back and forth between the data and the classification schema 
to verify the meaningfulness and accuracy of the categories and data placement (Patton, 1990). 

We identified six emergent categories from the year one coding which framed our year 
two activities. Following the collection of year two data, we coded and analyzed the data in 
precisely the same way as we did in year one. We coded data into six categories: target, plan, 
teach, assess, verify, and reflect. The final step was to identify major themes in each category 
and analyze themes across all categories. 
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FINDINGS AND DISCUSSION 

This two-year study generated a substantial body of data: 60 co-development team 
meeting transcriptions, six focus group interviews, individual interviews, three statewide retreats 
for all participants, journal entries, 250 electronic responses to focus questions, and various 
teacher-produced materials. These data document in great detail the journey of 44 high school 
teachers and higher education faculty from a more traditional classroom to a standards-based 
learning environment. 

While the year two data appeared to validate the cycle of teaching practices that emerged 
from year one data depicted in Figure 1, it also became clear that these practices did not occur as 
discrete actions. Teachers merged the practices into a repertoire that focused on student learning 
toward the target standard. For example, reflective inquiry, which was depicted as a culminating 
point in the cycle, occurred throughout the year and led to the continual monitoring and 
adjustment of practice, with the constant reference point being student progress toward the 
standard. Assessment results caused teachers to revisit how they planned instruction, to consider 
how they actually understood the target standard, and to question whether students had the 
necessary pre-requisite skills. As part of this study, teachers brought student work to their co- 
development team members and to statewide institutes for cross-scoring and verification of their 
judgments of student progress toward the standard. The process of anchoring to student work 
caused teachers to revisit more deeply practices throughout the cycle that might contribute to the 
improvement of student performance. 

For the purpose of this paper and to provide insight into our research question - what 
knowledge and skills are necessary for teaching in a standards-based system? - it is useful to 
begin with findings about the actual cycle of practices. These findings are presented in the 
following three categories that emerged from the data: Target /Plan, Teach/Assess, and Verify. 
We begin each section by defining the meaning of the practice(s) within a standards-based 
context, then describe specific applications and supply illustrative comments from STEP 
teachers. The recurring themes that arise across categories are highlighted in our concluding 
comments. 
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1. Target/Plan: Beginning with the End in Mind 

1.1 Meaning and Context 

Two interesting definitions of standards can be found in the American Heritage 
Dictionary (1996): 1) “A flag, banner, or ensign”... “in the army, a rallying point in battle,” and 
2) “an acknowledged measure of comparison for quantitative or qualitative value, a criterion, or 
benchmark, or yardstick” (p. 1752). 

Standards, in current educational trends, have become more than measurement tools; they 
are a “point to which to aspire” (Rothman, 1999, p. 39) that rallies together students and 
educators under the banner of achievement. Standards are clear and explicit. The standards target 
helps students and teachers know where they are going, the plan for teaching and assessment 
provides them with a roadmap to follow and indicates what they must do to get to their 
destination (Bodone, 2000). Together, teachers and students set out on the same journey 
“beginning with the end in mind” (Marzano, 1996). As Wolf and Reardon (1996) note: 

Just the presence of standards in classrooms is not transformative.... If the 
standards or new forms of assessment are to be tools for redistributing 
opportunities to learn, then they can’t have that inert, imposed quality. They have 
to be entered, used, and translated into students’ own language, (pp. 14-15) 

Our study indicates three teacher practices associated with targeting standard(s) and developing a 
plan for teaching and assessment. 

1.2 Application 

1.2.1 Develop an understanding that all teaching and learning begins and ends with the standard 

All the teachers in each of the six co-development teams targeted the same standard(s) for 
their discipline so that they would have a common context from which to work: (a) writing and 
literary analysis (English); (b) problem-solving in a sub-discipline such as algebra 
(mathematics); (c) scientific inquiry in a sub-discipline such as biology (science); (d) 
performance within a social/historical context in the arts such as music (visual and performing 
arts); (e) social science analysis in a sub-discipline such as history (social science); and (f) oral 
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proficiency and written expression (second languages). Conversations around the meaning of the 
targeted standard proved to be the most enduring throughout the second year of the study. 
Teachers indicated that the targeted standard: 

(a) focused as well as expanded learning; 

(b) was multi-dimensional, (i.e. it suggests content, process, and cognitive skills; connects to 
other disciplines, and has applications beyond the classroom); 

(c) suggested and defined assessment criteria. 

An English teacher commented on how the writing standard suggested the development of more 
complex cognitive knowledge and skills: 

[Students] understand writing as fulfilling an assignment. So, one thing that 
students need to know is how writing can be used to discover what they 
think — not what the teacher wants them to think. They must be able to apply the 
information they have learned about language structure to their own writing and 
thinking. They must be able to recognize clarity and coherence in their own 
writing and in the writing of others. They must be able to take risks and be 
allowed to experiment with different techniques. They must be able to write in 
clear sentences and paragraphs, unifying them into coherent pieces of writing that 
serve their own purposes in communicating their ideas to different types of 
audiences. 

1.2.2 Design a plan for teaching and assessment from the targeted standard 

Teachers often commented that the standard defines the planning process and focuses 
teaching and assessment on student learning. A high school science teacher noted that when one 
understood the meaning of scientific inquiry, the assessment of proficiency was evident: 

To meet the proficiency, I think students will need to be able to know what is a 
testable question or hypothesis. This is the foundation of the scientific process. 

Once they have developed the question, they need to come up with an acceptable 
experiment. Then, they can use the correct instruments or pieces of technology to 
get through the procedure. Again, I must stress that this is a process that must be 
modeled and practiced with students before they can ever be asked to do 
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something like this. They must be able to organize data, decipher the useable data, 
and discuss why it is useable. They should be able to present their findings, 
problems, challenges, applications, and a critique of their experiment in a 
meaningful way depending on the audience. 

Assessment is not necessarily a culminating event but woven all through the learning process. 
Over time, teachers described how student learning — not merely content acquisition — had 
become the target for assessment: 

While creative writing can show ability in many of the analytic writing traits, I 
feel that in order to meet the proficiency as it is currently written, students must 
put forth an idea which they have analyzed through their writing, thereby showing 
how they can discover and convey meaning in a written work. Though not stated, 
students need to be able to compose the piece on their own. I feel that to be 
considered proficient, the onus of responsibility of the development needs to be 
on the student. A metacognitive knowledge is needed about their writing. (English 
HS teacher) 



1.2.3 Involve students in the planning, design, and/or implementation of assessments 

Most teachers involved students in the translation of standards or the actual design of 
scoring guides. Benefits were realized in increased student involvement and engagement with 
learning. As one high school teacher commented: 

That is so rewarding. They understand so much more about the criteria, what you 
are looking for, that they will almost ask for it. They now know how to read it, 
they use it, they score with it, each other's. 

Whether or not students participated in the actual design of scoring guides (rubrics), 
using them in class helped students continually monitor their own progress toward the 
standard. As Andrade (2000) noted, 

. . .rubrics are also teaching tools that support student learning and the 
development of sophisticated thinking skills. When used correctly, they serve the 
purposes of learning as well as of evaluation and accountability. Like portfolios, 
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exhibitions, and other authentic approaches to assessment, rubrics blur the 
distinction between instruction and assessment. For this reason, I refer to them as 
instructional rubrics, (p. 13) 

The process of targeting a standard and planning instruction and assessment around that same 
standard allows for greater integration of teaching and learning. 

2. Teach/Assess: An Opportunity to Learn 
2.1 Meaning and Context 

Within the context of traditional evaluation, “assessment occurs after learning has already 
taken place” (Anderson and Speck, 1998, p. 6). Teachers in the initial stages of the study 
reported that classroom assessment usually followed teaching as a culminating activity. 
Furthermore, the occurrence of teaching and assessment activities might be more related to 
factors other than a clear standard for learning. For example, as one high school teacher explains, 
“I always teach Romeo and Juliet in the Spring; I do this unit when the books (materials) are 
available; we always do the questions at the end of the chapter.” 

As content coverage was replaced by a focus on standards, teachers came to view 
teaching and assessment as inextricably linked in a continuous process of expanding students’ 
knowledge and cognitive abilities. Assessment became “an opportunity to learn” (Baron and 
Wolf, 1996). Learning evolves rather than remains fixed and assessment serves to support 
students’ growth and increases their potential to develop proficiency (Bodone, 2000). This 
process of scaffolding “moves beyond static assessment of what is known to be a more 
interactive model looking at learning potential” (Gipps, 1994, p. 27). Gipps explains learning 
potential via Vigotsky’s idea of the zone of proximal development (ZPD which refers to “the gap 
between the actual development level as shown by the child’s unaided performance and her 
potential level as shown by her performance under adult guidance or collaboration with more 
capable peers” (p. 27). This complete integration of assessment with learning is based on the 
belief that students can and will become proficient (Bodone, 2000). 

Performance-based assessment is a type of evaluation used in standards-based education. 
In the United States, it is often regarded as any type of evaluation that is not multiple choice or 
standardized testing (Gipps, 1994) and associated with authentic or alternative assessment 
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(Darling-Hammond and Falk, 1997; Goodwin, 1997). The three approaches differ slightly, 
however: 

Alternative assessment is intended to distinguish [itself] from traditional, fact- 
based, multiple choice testing. Authentic assessment is intended to highlight the 
real world nature of tasks and contexts that make up the assessments. And 
performance assessment refers to a type of assessment that requires students to 
actually perform, demonstrate, construct, develop a product or a solution under 
defined conditions and standards. (Italics in original text) (Khattri and Sweet, 

1996, pp. 2-3) 

These three approaches embrace the earliest definitions of assessment. The origins of the 
word “assess” go back to a Latin verb assidere which means to sit by (ad- + sedere, to sit) 
(American Heritage, 1996, p. 111). Co-development team members eventually began to relate 
stories about “sitting by students” observing and estimating growth and looking for means to 
further enhance proficiency. A physics professor related his first time experience of pre- 
assessing students’ math ability within the first two weeks of class and discovering that they 
lacked the fundamentals to perform the more complex functions required in a specific physics 
inquiry project. This enabled him to “bring students along” rather than discover this lack of 
ability at the time of the mid-term exam. The integration of teaching and assessment as a process 
for enhancing student proficiency is an idea that may be unfamiliar to a large number of teachers. 
Many have been taught to assess students from afar, as objectively as possible, “standing over 
students” and evaluating “fixed knowledge and skills” rather than “sitting with them” to observe 
their growth and evolving cognitive skills (Bodone, 2000). 

During the second year of our study, when teachers were fully involved in implementing 
standards, six areas of practice emerged as contributing to teachers’ ability to integrate teaching 
and assessment in support of student learning. 

2.2. Application 

2.2.1 Begin teaching with an assessment of prior knowledge and experiences 

Teachers commented on how they consider students’ prior knowledge and experiences 
when they begin teaching or when they initially plan instruction. Teachers do not ascribe to the 
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tabula rasa notion that students come in as empty vessels to be filled. Instead, teachers build new 
knowledge from prior learning: 

In order to know what the students already know you have to ask them to respond 
orally or in writing to the following questions: 1) Do you know the name and the 
concept or definition? 2) Do you know the concept or definition but not the name? . 

3) Do you know the name but not remember the concept or definition? 4) Have 
you heard of this vaguely in your past, and if yes, in what context? 5) Have you 
ever heard of this before in your life? Begin with review. Find out what they 
know already and move on from there. Never assume. You can only grow from 
there. (Arts HS teacher) 

The assessment of prior learning in the initial stages of teaching includes learning about 
the student’s internalized experiences that may impact self-image and limit or enhance her/his 
possibilities (Bodone, 2000) in the standards-based learning environment: 

This is very significant learning, one that the teacher needs to be receptive to even 
through informal interactions. ... This view of self is strong and persistent and will 
set limits on further progress, sadly, even when it may appear that progress is 
being made.... It is the context of knowing the individual student, a context which 
should not be easily overlooked. (Loughran and Northfield, 1996, pp. 64-65) 



2.2.2 Build shared meaning of standards with students and design instruction toward these clear 
expectations 



When you first walk through the door, the standards-based classroom may not 
appear much different from any other classroom. But look closer. Something 
special is happening here. In an ideal standards-based classroom, there are no 
secrets about students’ academic responsibilities. The standards are posted on the 
walls of the classroom, as are rubrics showing the criteria forjudging student 
work and examples of student work that meets those standards. . .. The teacher 
spends time reviewing the standards at the beginning of the year and explains that 
these standards represent course goals.... Grading and assessment practices are no 
longer a mystery to students. Students must meet the criteria that match the 
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standards and reflect the course of study laid out in the syllabus. (Hampton, 1999, 
pp. 90-91) 

During co-development team meetings, we talked at length about the nature of standards 
and expectations. From the onset of their instruction, teachers clearly explain what they and the 
standard expect from students. Teachers commented that this increases the personal connection 
of students to their learning process as well as strengthens students’ intrinsic motivation to reach 
proficiency: 

Standards of excellence are set from the beginning and expectations are made 
known early in the semester. Students will connect with the standards initially by 
being apprised of the standards and the educational reform that has produced 
them.... I think it is a critical step to involve them in understanding the process of 
learning so that indeed they are responsible for not just the content piece but also 
understanding how learning happens for themselves and in general. Kids want to 
engage. (Arts HS teacher) 

To maximize the clarity of standards and students’ ownership of the process, teachers and 
students engage in “deconstruction” activities in which they work together to dismantle the 
standard, explain it, and then rewrite it in students' own words. This process helps to clarify 
expectations, put students at the center of their learning process, and make them more 
accountable for their work: 

After I analyze the standard and we talk about resources, the vocabulary of the 
criteria has to be made clear to the students, and they have to articulate what it 
means to them before we start. One of the things we learn as a group is to start to 
understand and agree on the meaning of these strange words that we use as 
English teachers like clarity and voice. I think that's a really important part of the 
process, to take any criteria, these booklets full of standards, full of teacher talk, 
and have the students own them, maybe even rewrite them in their own criteria 
language or something. (English HS teacher) 
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English teachers believe that knowing the standard inside and out and then creating 
strategies to reach it is how one should go about teaching— in other words build back from the 
standard to teaching: 

One way in which teaching to a standard differs drastically is that you focus on 
why you are teaching a particular book or project, not just what the book or the 
project is. Therefore, it is important to know the outcome of the unit first (which 
would be the standard), and then build backwards from there. It is necessary to 
backwards build, and then you determine the methods, strategies, and resources. 

What you have to begin with is the end, what do you want them to get out of this? 
Then that will determine methods, strategies, and resources. Even before the 
assessment, what's the standard? What is it that students should know? Then, 
what should students be able to do to show what they know? And I think that was 
a big shift, cause we've often attached assessment on the tail end. (English HS 
teacher) 

To talk about a standard in the abstract ...we're comfortable with (this), but we all 
have x number of years in experience and education. At 1 5 or 1 6, 1 don't know 
that I could necessarily have said off the top of my head what it is to 'analyze 
literature beyond a superficial level.'... There is still so much of this that is jargon, 
and in trying to explain it to the students, it's not necessarily in student language. 

So the PASS proficiencies are written for educators? If they don't even know what 
[a standard] would look like, they have no way of saying whether or not they meet 
it.... Anxiety loves structure, so people who are insecure, they want something 
clear, and it's our duty I think to make these standards clear. They're certainly 
complete by now, but they may not be as clear as they could be. (English HS 
teacher) 

2.2.3 Embed assessment into teaching 

Standards-based assessment focuses on process (formative) as well as outcome 
(summative) but primarily supports students’ growth and progress towards the proficiency 
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(Bodone, 2000). Through formative assessment, teachers can find out whether their instructional 
techniques are working with students or not, and make changes accordingly. Bell and Cowie 
(1999) define formative assessment as “the process used by teachers to recognize and respond to 
students learning in order to enhance learning, during the learning” (p. 198). This recognition 
and response involves close interactions between students and teachers who live together in the 
classroom and work together at reaching proficiency. Assessment is a constant feature of any 
activity which aims at analyzing progress of students towards being proficient. For arts teachers 
it helps to identify the authenticity of students’ work: 

On-going assessment helps the student to build daily in their portfolio collection 
and maintain ownership in the developmental process to strive towards meeting 
the proficiency. In class activities that provide on-demand tasks are necessary to 
ensure the work is their own and points are offered daily when working on 
extensive projects so much of the ground work is in place before the art work ever 
goes home. Verbal and written communication, as well as the visual, also serves 
as criteria when assessing individual’s authenticity. Criteria sheets, scoring 
guides, achievable benchmarks and student examples provide constant motivation 
and inspiration. (Arts HS teacher) 

2.2.4 Teach beyond the standard and further application of knowledge 

Reaching a standard is not the ultimate goal of standard-based teaching. Teachers 
communicated that their desire is to move their students beyond the standard to its direct 
application. Second language teachers, for example, want their students to get a more holistic 
understanding of what a language is made of and how languages compare with each other. They 
are further developing their students’ cognitive abilities: 

What I found very useful for them to do was to get them to explore enough of the 
language to think about journalizing and authoring a rule. And the tactic I’ll use 
sometimes is to say, “Well, let’s say you had-a very earnest non-native speaker of 
English who wanted to improve and this person was saying, well, how can you do 
that? And do you have a reason for that? And yet you do it. Y ou do it without 
thinking. (Second Language HS teacher) 
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Arts teachers also encourage their students to build knowledge beyond the standard and 
across the art forms they are studying to help students understand such concepts as creativity or 
artistic connection: 

The thing that I’m missing in a lot of these discussions, and in the real world for 
that matter is the interconnectedness of what we do. At the end of every trumpet 
lesson, my question of the week is, I take a piece of visual art and I put it on a 
stand in front of them, and I say: ‘What’s going on in this picture?’ And it makes 
them think entirely differently about art, because this is something that has 
nothing to do with music. Now we are talking about, ‘How do you feel?.... You 
know, when you play the trumpet, you have to make people see this too. You 
have to make people feel what is going on in this music. Just like the artist had to 
make people feel when they see this picture. You don’t get the benefit of pictures. 

You just have notes. How are you going to make that feeling come across?’ (Arts 
HS teacher) 

Science teachers put forth the notion that students are proficient when they are able to 
connect scientific inquiry to everyday life/situations in what might be termed informed 
application of scientific concepts: 

I believe that for a student to truly demonstrate proficiency, they must be able to 
take the information they have learned and show that they can apply this 
knowledge to a new situation. I believe that it is only when they have escaped the 
possibility of using memorized answers, or use phrase recognition to solve a 
problem, that they actually demonstrated true proficiency. I don't expect all 
students will necessarily design and conduct a conventional science lab 
experiment from the ground up. It would be sufficient if they could apply a basic 
scientific approach to everyday questions that are open to scientific inquiry. This 
implies that they can distinguish issues and questions that are within the domain 
of scientific inquiry from those that are not. A student who is proficient in this 
area would not only be able to design a scientific approach to a given question 
when asked to do so, they would also exhibit a tendency to apply this approach to 
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decision-making when confronted with scientifically valid questions by their 
everyday experience. (Science HS teacher) 

While a standard requires focused instruction, it still allows for expansion into more 
informed and complex performances that require students to think beyond what is being taught. 

2.2.5 Respond to students’ varying levels of ability 

Teachers use various techniques such as pre-testing to assess students’ prior knowledge 
and varying levels of ability. Instructional design begins with what students know and progresses 
from there, employing strategies such as peer support and cooperative learning. However, 
teachers recognize the challenge of addressing varying levels of ability and their own limitations, 
at times, to satisfy every need: 

No two people ever come to a situation with the same levels of knowledge and 
skills! I assume this to be true in any classroom. Luckily, with writing, the teacher 
approaches each paper individually. Working with writing is like writing a series 
of letters back and forth and I get to watch the students grow. We create a 
pyramid of writing, an idea I stole from an elementary teacher. The students 
themselves construct it, based on how they respond to the standard, what things 
they do well, where they need help. Each paper is read by a mechanic expert, a 
transition expert, a parallel structure expert, a clarity expert, a focus expert, and 
organizational expert, etc. specifically for the aspect for which the expert is 
responsible. There is much writing, much coaching on each paper, so by the time 
the student turns in a draft to me, he's been able to eliminate many problems and 
let me read for ideas, balance, insight, and diction. The student knows 
immediately who needs extra help and prodding, and insures they receive it.... 

Wouldn't it be rare that you would have a homogeneous writing group? I think 
you're going to have various levels of skills and different components of writing, 
so it's just sort of a normal thing to walk in and know that you're going to have to 
teach each writer using that person's writing as a tool. To assume they'd all be at 
the same place... it would be rare. (English HS teacher) 
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The skill, training, and ability of teachers to inspire, direct, and redirect students’ thinking about 
learning, is critical to appropriately address varying levels of ability. 

2.2.6 Define proficiency and sufficiency through multiple demonstrations 

Demonstrating one’s performance through multiple opportunities is another method that 
teachers used. Multiple demonstrations include a variety of assessments, (i.e. reflective pieces, 
on-demand tasks, group projects, portfolios). This helps teachers in their exploration of what 
makes a student’s performance proficient. When we asked teachers assessment questions (see 
Table I), across the content areas, they spent considerable time debating on definitions of 
proficiency, sufficiency, and consistency. Consider the words of English and Arts HS teachers: 

I don’t think we can talk about the idea of ‘knowing’ proficiency without also 
talking about consistency and sufficiency. The determination of proficiency 
cannot be based on one task a student did at one time; secondly, one assignment 
alone, no matter how broad in scope, is not sufficient to determine proficiency. 

Because we all know students leam in different ways, we allow for that by giving 
the student a variety of ways to meet the proficiency. My point is we will know a 
student is proficient when they have consistently demonstrated the level of skills 
and knowledge we are expecting on a variety of assignments. One performance 
on one day on one task does not tell us anything about the students’ skills other 
than what they were able to do on one day. (English, HS teacher) 

Proficient or not proficient? That’s not the only question! When asked to think 
about proficient student performance and how that is measured, I want to balk, I 
want to say that Art cannot be measured and that creations erupting from an 
artist’s soul are much more difficult to measure than the validity of a geometry 
proof. I am right that it is difficult to measure, but the more I work with standards- 
based education, the more I realize that proficiency can be measured. Can 
proficiency be assessed in an extracurricular context? What might proficient 
demonstrations look like? How many roles constitute sufficient and proficient 
collections in the Theatre Arts? (Arts HS teacher) 
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2.2.7 Shift teachers’ and students’ roles in the learning process 

Teachers reported that over the two years of the study, they gradually shifted their role 
from dispensing knowledge to guiding and facilitating student learning toward proficiency. By 
choosing different teaching strategies, teachers evolved into the role of guiding cooperative 
learning activities, monitoring student progress, and, in some cases, trusting students to teach 
each other via peer reviewing and tutoring: 

In cooperative learning, the teacher has to kind of orchestrate; the kids have a 
tendency to talk with one another. The idea is for the kids to share their thoughts 
and feed off each other. The problem here is that when you have... the kind of 
cliquish situations where the elevated kids or the kids with excellent skills work 
together and goof off. So, orchestrate situations where you have two kinds of kids 
working together. That can work effectively in terms of engagement. (Social 
Science HS teacher) 

If you’re also interacting with the student as a learner and as a guide, then you 
have to — it’s more like what we do in lab, where you turn students loose and let 
them do something and then you try to observe and correct error and to make 
suggestions about how they could improve their work. Ask leading questions 
about what they’re doing, why they’re doing it, ask them to explain, recite, all 
those things.... Does the mode of delivery change? I guess the teacher’s role 
changes to one of facilitator, coach, guide, and observer. (Science HS teacher) 

English teachers reported involving their students in various interactive group activities 
where students’ expertise is required and valued to review, assess, and critique each other's 
writing. Again, the teacher moves away from being the sole authority to becoming a participant 
and facilitator of students' learning. After group reviews, students are given a chance to rewrite 
in order to apply what they have learned through the review process. This invites greater risk- 
taking on the part of the student as illustrations from their own writing are used for group review 

After students turned in their papers, I copied several of their sentences and 
paragraphs onto a transparency and showed them (anonymously) to the class. As 
a class we critiqued these examples and rewrote them. The class found this 
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enjoyable and beneficial since many of them made the same mistakes. They were 
given the opportunity to revise their writing. Students already bring to the 
classroom a sense of what makes writing clear or acceptable. They have a 
conscious awareness of coherent prose (whether they want to admit it or not). 

While reticence occasionally inhibits their willingness to address their peers' 
drafts thoroughly, students often know when an essay is really inadequate. I hand 
out sample student essays from previous terms, and we analyze the characteristics 
of good writing in these essays. In addition, we work in groups to help apply 
editing and reading techniques. These groups must complete rigorous response 
sheets in order to analyze each other's drafts. Sample essays can also show 
students the goal. I'll hand out essays that do not meet the proficiency, discussing 
with them writing that is not yet acceptable. A gradual refining of their 
observational and analytical skills helps students identify the specific qualities of 
clear, effective writing. They do not always agree with my selections, either. They 
question why an essay might get a high score when they feel it isn't well 
supported or convincing. (English HS teacher) 

As students and teachers together deconstruct and reconstruct standards and scoring 
guides, students do not change the content of proficiencies. What differs is the form in which 
standards are presented. It now corresponds to students’ contexts as defined by students. The 
teachers who facilitate and guide students through this elaboration process contribute to the 
development of students’ voices in the classroom (Bodone, 2000), or what Kreisberg (1992) 
refers to as empowerment, “a social process of self-assertion in one’s world” (p. 19). 

3. Verify: Confirming Judgments of Proficiency 
3.1 Meaning and Context 

Webster’s dictionary ( 1963) defines verify as “to prove to be true... or accurate,... to 
substantiate, ... to confirm” (p. 2543). The traditional classroom and daily high school or 
university routine have not afforded teachers the opportunity to verify, to share student work and 
confirm judgments of student proficiency in meeting a standard. Rather than engage in collegial 
activities around student learning, teachers are more likely to experience what one social studies 
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teacher referred to as “a splendid isolation.” This isolation is antithetical to a standards-based 
environment that is based on a common understanding of standards and consistent interpretation 
of student performance. 

In a standards-based system and particularly in Oregon, teacher judgment and the ability 
to verily judgments play a central role in the assessment systems for both K-12 and college 
admission. Beginning in 2005, the Oregon University System (OUS) will base admission to the 
seven public institutions on assessment information generated through a combination of 
performance assessment (collections of student work in specific standards) and multiple choice 
tests (national and state). During the development of this Proficiency-based Admission Standards 
System (PASS), high school teachers are responsible for teaching to the standards and working 
with students to generate collections that are sufficient (where the range of work addresses the 
criteria for meeting the standard) and proficient (where the performance is at the level necessary 
for college entry coursework). Verification is the process of confirming individual teacher 
judgments of proficient performance (evidenced in the collections of student work) with the 
judgments of colleagues. Teachers participating in the development of PASS may verily their 
judgments of collections of student work with other teachers in their schools and with other 
teachers from around the state at PASS Verification Institutes or, in the case of teachers in this 
study, with co-development team members. 

During this two-year study, the verification of teacher judgment of student proficiency 
became both a formative and summative process. During co-development team meetings, 
participants focused their conversations on the PASS standards in their content area. They 
described how they were designing teaching and learning activities around the target standard 
and how they were working with students to generate a variety of “evidence” that, when 
collected together, provided a clear picture of proficient performance. As trust developed among 
team members, teachers shared classroom tasks, scoring guides, and samples of student work. 
Teachers requested specific feedback on their classroom materials, planned activities, strategies, 
and approaches, and as to whether or not students appeared to be approaching proficiency. The 
PASS Verification Institutes, held in February and May of each year, provided a summative 
experience in which co-development team members’ collections of student work were included 
with those of over 125 other teachers, and judged on the merits of sufficiency of evidence and 
proficiency of performance. 
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As PASS is phased in over the next five years to full implementation, it will be critical to 
have consistency of judgment around the high-stakes decisions surrounding college admission. 
Moving beyond the notion of simple consistency of judgment, Gipps (1994) calls for 
“comparability, that is, equivalence in the application of standards across assessment tasks, 
assessment occasions, students, classes, schools and school years” (p. 28). Furthermore, this 
degree of consistency requires “shared experience and understandings within a community of 
judgment” (Wolf, 1995, p. 71). In other words, this common understanding must be embedded in 
the minds and professional practice of teachers. In our study, the following four areas of practice 
emerged as contributing to teachers’ ability to verify judgments of student proficiency. 

3.2 Applications 

3.2.1 Understand the standard and apply criteria for judgment 

We found that across all six content areas, teachers spoke to the need of having an 
accompanying set of criteria that would guide their judgment toward an understanding of 
whether or not the student has demonstrated proficiency. These criteria promote a common 
vision of student performance and ensure that teacher judgments are based on this common 
vision rather than on idiosyncratic preferences. In the content areas for which the PASS criteria 
are currently developed in the greatest detail (English, math, and science), comments from high 
school (HS) teachers and higher education (HE) faculty attest to the link between specific 
guidelines and consistent judgments: 

If we can come to a common ground first before even looking at a student work, 
and if the scoring guide is clear enough to be used in justifying one's assessment, 
then it is possible to come to an agreement. This process will need to be 
maintained. We tend to adapt our expectations to what we see; therefore, when 
teachers assess their own work without any opportunity for ‘refocusing,’ then they 
may be straying from what is generally agreed upon as proficient performance. 

When the classroom teacher and an outside evaluator (from another school) can 
agree, then the student is clearly proficient. (English HS Teacher) 
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When I score essays, I have to admit that I look for what the essay does well. 
Several of the other readers look for an accumulation of problems, and after a 
certain (unspecified) number, score the essay as ‘not proficient.’ I have done 
‘mini-reads’ with one or two faculty this year, as well, and we often agree on 
scores. This comes from discussing in a more focused way the criteria for scoring 
and the expectations we have for good or exemplary performance. There are 
certain faculty, though, who do not agree with me on standards and criteria for 
grading. They seem to want to give credit for ‘effort,’ which drives me crazy. 
(English HE Professor) 

When I attend a Verification Institute, I am amazed how much correlation of 
judgment there is between my judgments and those of other teachers. I know that 
there is a common understanding of the standards and that teachers understand the 
need to be consistently critical of student work in regards to standards. (Science 
HS Teacher) 



3.2.2 Apply scoring guides and illustrations of proficient performance to student work 

Across all six content areas, we found teachers embraced the use of various types of 
scoring guides that called for various approaches for arriving at a judgment. A formal state- 
adopted instrument, like the Oregon writing scoring guide, requires judgment on each of six 
analytical traits. Less formal guides include the PASS proficiency standard scoring guides, which 
provide detailed criteria at the “meeting proficiency” level. PASS calls for a holistic judgment 
about whether or not these criteria are represented in the student work directly, or whether 
sufficient evidence is presented for the rater to infer that the student would be proficient in all 
criteria. Informal scoring guides created by teachers were also shared throughout the duration of 
STEP. Regardless of the type of scoring guide or judgment called for, teachers acknowledged 
the value of these tools for enhancing common understanding and tempering individual variation 
in judgment (or particular biases) towards one’s own students as evidenced in these remarks: 

I will know that I share a common understanding when we can look at work and 
‘score’ it or grade it within one grade or number, using exactly the same scoring 
mechanism and making sure we understand the mechanism the same way. There 
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is a universe of interpretation that comes into the grading or evaluating activity, 
though, and one of the most important steps for me is to make sure everyone 
involved understands the scoring guide the same way. When I am confident that 
we all understand the scoring guide the same way, then the activity can proceed 
without too much arguing about the features we're looking for. (English HE 
Professor) 



Scoring guides have allowed me to understand and implement a consistent 
evaluation process which in turn has given me a common understanding of 
student proficiency. (Social Science HS Teacher) 



I most often agree with colleagues who have knowledge of the scoring guides. 
(Mathematics HS Teacher) 

Along with scoring guides, exemplars of student performance complete the “picture” of 
student proficiency, as confirmed by a participant: 

I think any "common understanding" of standard will have to be re-negotiated 
with every reading and assessment experience. The value of "range finders," or 
sample papers that resemble features found in the whole collection of samples is 
high in this kind of system. If we have the standard and continually refer to it, 
that is good. What tends to happen, however, is that we "drift" even within a 
single-sitting assessment experience. Heaven knows what happens over time and 
with many different age levels and types of work to assess. Thus, experiences 
where we get together to evaluate and discuss the same work, to agree on the 
salient features, and then to read and assess will be vital. (English HE Professor) 

The use of scoring guides and exemplars were frequently related in teachers’ comments 
about the purpose of the judgment being made. For PASS, teachers are asked to judge the 
readiness of the student to do entry-level college coursework. This level of readiness, in general, 
was particularly difficult to define in the arts where clear exemplars and defined performance 
levels are lacking. High school Arts teachers explained how they addressed this challenge when 
immersed in the verification process by shifting the focus to their vision of the archetypal 
entering student to reach common understanding of proficient performance: 
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I think we were much more realistic about our expectations this year. We used 
"Joe Football" as the guide to help us remember that although we are attempting 
to raise the bar, there are many students who are quite average (or even 
untalented) in the arts. We cannot expect every college bound student to hold the 
lead in a play and design costumes and know theatre history, all at a mastery 
level! (Arts HS teacher) 

While scoring guides and exemplars of student proficiency provided the tools for 
reaching common judgment, they are still the means to an end - the verification process. As 
stated by a second language teacher: 

The first time I have participated in an actual cross-scoring of student work was at 
the PASS Verification Institute. This was perhaps the most exciting workshop I 
have attended to date. To me, it was significant and FINALLY we were sitting 
down with scoring guides in hand, however crude the scoring guides may be, and 
holding student work up to a common standard. I found this to be excellent and 
stimulating. I would have liked to spend an entire week doing this work. 



3.2.3 Work with colleagues to enhance judgment, expand assessments and deepen understanding 

As professional working relationships were developed across the six co-development 
teams, a genuine appreciation for collegial activity emerged as well as a recognition that this type 
of activity rarely occurs in schools as they are currently structured. Teachers who are isolated by 
virtue of their discipline, (e.g. the sole drama teacher) welcomed the opportunity to join a larger 
statewide collegial group. These collegial activities are purposive, providing a vehicle for 
enhancing teacher judgment, expanding frameworks for assessment and deepening common 
understanding of proficiency that will ultimately impact classroom practice. Teachers in several 
disciplines discussed their collegial work on standards as illustrated by these statements: 



Talking with peers is the way I think all of us will know when we are making 
appropriate judgments. This is new to many schools since I usually go for a whole 
day without seeing another teacher. This "talking with peers" thing is a 
revolutionary concept especially since there is only one of me in this district and 
very few others understand that theatre is more than fancy costumes and cool sets. 
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We need to make the dialogue with the other Arts teachers meaningful; we need 
to speak a common language. (Arts HS Teacher) 

I will know we share a common understanding when we all sit down with the 
same set of goals (that is, a genuine proficiency orientation), and proceed to 
hammer out the curricular system for our program, constantly proposing and 
judging with reference to language proficiency. (Second Language HE Professor) 

The experience with the co-development team caused several individuals to envision how 
they might tackle the challenge of introducing collegial learning in the context of their own 
school or campus. It is interesting to note that there is always a ‘standards focus’ to this collegial 
activity, to the verification process, to reviewing student work, and to improving comparability 
of judgment. As the following participants stated: 

I will try to work with colleagues in my discipline to work up a common scoring 
guide so that students throughout the school know that teachers are consistent. 1 
will work with teachers in different disciplines to reinforce the concept of 
standards and to reinforce the use of scoring guides, and the relevance and 
connectedness of various disciplines. I will ask to score work from other 
resources in the discipline, and have other teachers score my student work to help 
verify judgments about student proficiency. (Social Science HS Teacher) 

Faculty, too, even when they feel insecure about "judging" student writing or 
speaking, can gain confidence when they share their insights in a group. 

Respecting each other's ideas about student performance is one of the best ways to 
get faculty to share their students' work. One of the most important ways to get 
faculty to verify their judgments is by making them feel secure enough to share 
their students' work." Another way would be for faculty to sit in on each other's 
student conferences and listen to what features of writing or speaking or critical 
thinking get emphasized in these meetings. This might be a good "mentor" model 
for establishing consistency. (English HE Professor) 

3.2.4 Develop content knowledge and expertise 
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Across all content areas, the development of teachers’ content knowledge was frequently 
cited as a vital factor in reaching comparable judgments, as this high school arts teacher explains: 

I have found judgments are comparable especially when the breadth and depth of 
the knowledge base is similar with those in collaboration. The quality of 
assessment and effectiveness of scoring depends largely on the knowledge of the 
group comparing the student's work. 

In addition to content knowledge as we traditionally understand it, teachers referred to a 
broader expertise marked by a deep understanding of teaching and learning in one’s discipline, 
defined by Schulman (1987) as “pedagogical content knowledge.” This notion was seen as a 
necessary attribute of teacher professionalism that contributed to comparable judgments. STEP 
teachers noted an interesting aberration in scoring: teachers who lacked knowledge tended to rate 
student proficiency at a higher level. A few examples of this observation are included in the 
following statements: 

If I am with teachers who know mathematics, then we are pretty consistent. If I 
am with a group of teachers who have not kept up with mathematical education 
over the last 10 to 20 years then they tend to have higher scores on papers. 

(Mathematics HE Professor) 



I know that when my student teacher began scoring writing samples, he gave 
everyone 5 and 6 when I was giving them 2 and 3. It took him a lot of practice 
and discussion. He admitted that it 'sounded good' or 'that the student really meant 
something else.' This is where the standards are so important: Could the student 
perform at a specified level? Did we set a predetermined level of achievement 
before scoring? Did the student have prior knowledge of how they would be 
assessed so that they could have opportunity to reach the minimum level of 
achievement? (Second Language HS Teacher) 

What I worry about is that many teachers do not have the understanding (as 
opposed to knowledge) of content needed to have assimilated professional norms, 
and when faced with real students in a real life situation, they will allow their 
judgment to be clouded by a myriad of external factors, just as it happens with 
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grades in the current system. We MUST build in external validation and quality 
assurance to make this work! (Mathematics HE Professor) 

These four areas of practice suggest that the opportunity to verify one’s judgment of student 
performance with one’s colleagues provides teachers with continuous professional development 
and deepens content knowledge as well as pedagogy. 



CONCLUSION 

The findings from the STEP study suggest that teachers engaged in the actual 
implementation of standards provide clear imperatives for those charged with the formulation of 
educational policy and the design of teacher-preparation and professional-development 
programs. Our experience taught us that teachers are the primary agents of school reform; 
therefore, they should be included as co-developers in all stages of design, implementation, and 
continuous improvement of the standards-based system. Other educational researchers also 
support this finding (Darling-Hammond, 1990; Hong, 1996; Shedd and Bacharach, 1991), as do 
policy makers who are attempting to implement successful reforms in schools nationally and 
internationally. The International Commission on Education for the 21st Century (1996), in its 
report to UNESCO, noted the following: 

No reform can succeed without the co-operation and active participation of 
teachers. Attempts to imposed educational reforms from the top down, or from 
outside, have obviously failed. The countries where the process has been 
relatively successful are those that obtained a determined commitment from local 
communities, parents, and teachers, backed up by continuing dialogue and various 
forms of outside financial, technical, or professional assistance, (p. 29) 



As our teachers had the opportunity to truly immerse themselves in the co- 
development and implementation of standards-based reform, they reminded us that the price 
of exclusion is costly. Their professional expertise, and that of other PASS teachers, has 
contributed to the continuous improvement of the PASS framework of standards, 
assessments, criteria, scoring guides, and professional-development activities. Teachers’ 
experiences have influenced the successive iterations of PASS standards, scoring guides, and 
assessment guidelines. We have welcomed the prior experiences, knowledge, and personal 
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philosophies that teachers have brought to the co-development effort. We have encouraged 
them to share what they deem to be relevant to inform our discussions and the development 
of PASS. 

The major findings of this study have also informed the next steps in the development 
of standards-based reform in Oregon. The 1 5 areas of teacher practice presented here under 
the three major categories— target/plan, teach/assess, and verify— served as the foundation for 
a series of eight training modules used across 65 PASS high schools in the 1999-2000 
academic year. This program, entitled Teachers Teaching Teachers (T3), is proving to be 
highly successful as a professional development tool. Over 1 50 English, math, and science 
teachers who are part of a statewide collegial research and development group are targeting 
specific standard(s), planning instruction and assessment around those standards, and 
integrating teaching and assessment to produce collections of student work. In February 
2000, teachers submitted their collections of student work to their colleagues at a statewide 
Verification Institute. The effectiveness of the training may be measured by the degree to 
which teachers from different parts of the state produced sufficient collections of student 
work that were judged at the same level of proficiency by their colleagues (68% same rating, 
94% adjacent score rating). These preliminary results would seem to indicate the presence of 
“shared experience and understandings within a community of judgment” (Wolf, 1995, p. 

7 1 ). 

The framework of teacher practice presented here will be further refined for inclusion 
in the teacher preparation and licensure programs at Oregon’s 16 teacher-education 
institutions. Beginning teachers and teachers engaged in continuing professional 
development will benefit as they develop expertise in the implementation of standards. 

The practices shared here, when considered individually, agree with the classic work 
of Rosenshine (1976) and Brophy (1979) on teachers’ effects on student performance that 
heavily influenced current teacher-training programs. The challenge of the standards 
movement, however, is that these practices must be woven together to support profound 
shifts in thinking suggested by our study. These shifts include: (1) all educational 
professionals need to share a common understanding of student learning (as defined by 
standards); (2) assessment practices must be transformed to more closely allow actual 
demonstration of student knowledge and skills; and, finally, (3) teacher and student roles 
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must more clearly reflect a facilitative style that encourages the development of complex 
knowledge and skills. The development of a framework of teacher knowledge and skills 
necessary in a standards-based system is simply one step in this larger process of 
implementation. 
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